Random renoised test DataSet of Mnist, $10k$ examples.
Test DataSet of Mnist concatenate with diferent noise levels probabilities from $0\%,10\%, \cdots ,100\%$, $11 \times 10000 = 110k$ examples.
While evaluating renoise the prediction of the first autonecoder before the prediction of the second autoencoder. The results where worse.
For the metrics the ouput $y \in [-1,1]$ was trasformed to $\hat{y} \in [0,1]$ and then rounded to two decimal places, since we want to better observe the dieffernce betwen runs instead of the absolute value.
For the classifier the data was not converted.
absolute $\epsilon$-accuracy
diff <- |target_set - predicted_set| // pixelwise difference stored as (10000, 784)
accu <- 0 // accumulator
loop elem in diff // for each element in diff i.e. for each number, image (784,)
accu <- |{i ∈ elem : elem < ε }| / 784 //count how many elements are > ε and average over pixels, i.e. divide by 784
accu <- accu/10000 // Average over examples in image set
squared $\epsilon$-accuracy
diff <- (target_set - predicted_set)^2 // pixelwise difference stored as (10000, 784)
accu <- 0 // accumulator
loop elem in diff // for each element in diff i.e. for each number, image (784,)
accu <- |{i ∈ elem : elem < ε }| / 784 //count how many elements are > ε and average over pixels, i.e. divide by 784
accu <- accu/10000 // Average over examples in image set
evaluations
$\operatorname{mse} =\frac{1}{m} \sum_{j=1}^{m} \frac{1}{n} (Y_j-\hat{Y_j})^2 = \frac{1}{m} \sum_{j=1}^{m} \frac{1}{n} \sum_{i=1}^{n} (y_i-\hat{y_i})^2$
where:
$\operatorname{mae} =\frac{1}{m} \sum_{j=1}^{m} \frac{1}{n} |Y_j-\hat{Y_j}| = \frac{1}{m} \sum_{j=1}^{m} \frac{1}{n} \sum_{i=1}^{n} |y_i-\hat{y_i}|$
where:
Average over: the maximal absolute diference between pixels in each example. i.e. $\underset{i}{mean}(max(|\mathbb{Y}_i-\mathbb{\hat{Y}}_i|))$
Maximum over: the maximal absolute diference between pixels in each example. i.e. $max(max(|\mathbb{Y}_i-\mathbb{\hat{Y}}_i|))$
where:
The test data that was used in validation for the training was renoised in a random mixed form while the input data for training was renoised with an equal noise lvl, with the idea of simulating real world case for the validation. Butin the iteration renoising the prediction of the validation data and the input data was equaly renoised instead of the validation set being randomly renoised.
Acc means equal pixels are more but unequal pixels have a higher difference.